{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 使用RNN来生成莎士比亚风格的句子\n",
    "\n",
    "这个例子会用莎士比亚的著作来训练一个char-level RNN语言模型，同时使用它来生成莎士比亚风格的句子。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 数据准备\n",
    "\n",
    "输入文件是纯文本文件，我们会使用unidecode来把unicode转成ASCII文本。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "file_len = 1115393\n"
     ]
    }
   ],
   "source": [
    "import unidecode\n",
    "import string\n",
    "import random\n",
    "import re\n",
    "\n",
    "all_characters = string.printable\n",
    "n_characters = len(all_characters)\n",
    "\n",
    "file = unidecode.unidecode(open('../data/shakespeare.txt').read())\n",
    "file_len = len(file)\n",
    "print('file_len =', file_len)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这个文件太大了，我们随机的进行截断。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "re lay hold of him;\n",
      "Bear him to the rock Tarpeian, and from thence\n",
      "Into destruction cast him.\n",
      "\n",
      "BRUTUS:\n",
      "AEdiles, seize him!\n",
      "\n",
      "Citizens:\n",
      "Yield, Marcius, yield!\n",
      "\n",
      "MENENIUS:\n",
      "Hear me one word;\n",
      "Beseech you, tr\n"
     ]
    }
   ],
   "source": [
    "chunk_len = 200\n",
    "\n",
    "def random_chunk():\n",
    "    start_index = random.randint(0, file_len - chunk_len)\n",
    "    end_index = start_index + chunk_len + 1\n",
    "    return file[start_index:end_index]\n",
    "\n",
    "print(random_chunk())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 定义模型\n",
    "\n",
    "回忆一下之前的Char RNN 分类器，我们是“手动”实现的最朴素的RNN，现在我们使用更加先进的GRU。\n",
    "另外之前是没有Embedding的，直接用字母的one-hot作为输入。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.nn as nn\n",
    "from torch.autograd import Variable\n",
    "\n",
    "class RNN(nn.Module):\n",
    "    def __init__(self, input_size, hidden_size, output_size, n_layers=1):\n",
    "        super(RNN, self).__init__()\n",
    "        self.input_size = input_size\n",
    "        self.hidden_size = hidden_size\n",
    "        self.output_size = output_size\n",
    "        self.n_layers = n_layers\n",
    "        \n",
    "        self.encoder = nn.Embedding(input_size, hidden_size)\n",
    "        self.gru = nn.GRU(hidden_size, hidden_size, n_layers)\n",
    "        self.decoder = nn.Linear(hidden_size, output_size)\n",
    "    \n",
    "    def forward(self, input, hidden):\n",
    "        input = self.encoder(input.view(1, -1))\n",
    "        output, hidden = self.gru(input.view(1, 1, -1), hidden)\n",
    "        output = self.decoder(output.view(1, -1))\n",
    "        return output, hidden\n",
    "\n",
    "    def init_hidden(self):\n",
    "        return Variable(torch.zeros(self.n_layers, 1, self.hidden_size))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 输入和输出"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "每个chunk会变成一个LongTensor，做法是遍历每一个字母然后把它变成all_characters里的下标。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([ 10,  11,  12,  39,  40,  41])\n"
     ]
    }
   ],
   "source": [
    "# 把string变成LongTensor\n",
    "def char_tensor(string):\n",
    "    tensor = torch.zeros(len(string)).long()\n",
    "    for c in range(len(string)):\n",
    "        tensor[c] = all_characters.index(string[c])\n",
    "    return Variable(tensor)\n",
    "\n",
    "print(char_tensor('abcDEF'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "最后我们随机的选择一个字符串作为训练数据，输入是字符串的第一个字母到倒数第二个字母，而输出是从第二个字母到最后一个字母。比如字符串是\"abc\"，那么输入就是\"ab\"，输出是\"bc\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "def random_training_set():    \n",
    "    chunk = random_chunk()\n",
    "    inp = char_tensor(chunk[:-1])\n",
    "    target = char_tensor(chunk[1:])\n",
    "    return inp, target"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 生成句子\n",
    "\n",
    "为了评估模型生成的效果，我们首先需要让它来生成一些句子"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "def evaluate(prime_str='A', predict_len=100, temperature=0.8):\n",
    "    hidden = decoder.init_hidden()\n",
    "    prime_input = char_tensor(prime_str)\n",
    "    predicted = prime_str\n",
    "\n",
    "    # 假设输入的前缀是字符串prime_str，先用它来改变隐状态\n",
    "    for p in range(len(prime_str) - 1):\n",
    "        _, hidden = decoder(prime_input[p], hidden)\n",
    "    inp = prime_input[-1]\n",
    "    \n",
    "    for p in range(predict_len):\n",
    "        output, hidden = decoder(inp, hidden)\n",
    "        \n",
    "        # 根据输出概率采样\n",
    "        output_dist = output.data.view(-1).div(temperature).exp()\n",
    "        top_i = torch.multinomial(output_dist, 1)[0]\n",
    "        \n",
    "        # 用上一个输出作为下一轮的输入\n",
    "        predicted_char = all_characters[top_i]\n",
    "        predicted += predicted_char\n",
    "        inp = char_tensor(predicted_char)\n",
    "\n",
    "    return predicted"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 训练"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "一些工具函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "import time, math\n",
    "\n",
    "def time_since(since):\n",
    "    s = time.time() - since\n",
    "    m = math.floor(s / 60)\n",
    "    s -= m * 60\n",
    "    return '%dm %ds' % (m, s)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "训练函数："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "def train(inp, target):\n",
    "    hidden = decoder.init_hidden()\n",
    "    decoder.zero_grad()\n",
    "    loss = 0\n",
    "\n",
    "    for c in range(chunk_len):\n",
    "        output, hidden = decoder(inp[c], hidden)\n",
    "        loss += criterion(output, target[c:c+1])\n",
    "\n",
    "    loss.backward()\n",
    "    decoder_optimizer.step()\n",
    "\n",
    "    return loss.item() / chunk_len"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来我们定义训练的参数，初始化模型，开始训练："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0m 9s (100 5%) 2.3245]\n",
      "Whell! thhe thihe hine thy sher as, wof he's ghe liths.\n",
      "\n",
      "OLIU:\n",
      "HIO:\n",
      "What hamedt juta sthithe mapnth ni \n",
      "\n",
      "[0m 19s (200 10%) 2.1475]\n",
      "Whode.\n",
      "\n",
      "aor whath a mater but amt ther nourdt do in to his in for ante, wor be the thow kitake thing i \n",
      "\n",
      "[0m 28s (300 15%) 2.0459]\n",
      "What him besty, be\n",
      "tis thou that tith thel and to say beave wourt to ilw: spearsteres ye com. entry,\n",
      "W \n",
      "\n",
      "[0m 37s (400 20%) 2.0369]\n",
      "Whad may be his tain,\n",
      "Fims in at in I prane oar not, it im\n",
      "Of shim; mastruede.\n",
      "\n",
      "CORK:\n",
      "And me me san, g \n",
      "\n",
      "[0m 47s (500 25%) 2.2005]\n",
      "Whom mans.\n",
      "\n",
      "Thathst sace, his perewill sold nome diest, sul. and ligule\n",
      "The as aly moner\n",
      "And Hadly jef \n",
      "\n",
      "[0m 57s (600 30%) 1.9203]\n",
      "Whath my my thour.\n",
      "\n",
      "ThING ERDIAND:\n",
      "To my cer,\n",
      "And must my lentit.\n",
      "\n",
      "GLOUCESTER:\n",
      "I deast was than and be \n",
      "\n",
      "[1m 6s (700 35%) 1.9028]\n",
      "Whien at hepicter,\n",
      "For thee corron the to hy so, we at loves, king the he mastingly his door preist, a \n",
      "\n",
      "[1m 15s (800 40%) 1.9324]\n",
      "Whtild that mition and on him erears flature'd for um deancue though with for that which four adgain H \n",
      "\n",
      "[1m 25s (900 45%) 1.9309]\n",
      "Whall with him for of the lookine\n",
      "Do allow, to I dook son! Edenater to dence\n",
      "To this preato sents then \n",
      "\n",
      "[1m 35s (1000 50%) 2.0301]\n",
      "Whave of the speince to that sole do and iter,\n",
      "The the bant as by and it hinsss it thour gronighter ar \n",
      "\n",
      "[1m 45s (1100 55%) 1.9934]\n",
      "What they intremice wet ware, his why, yours to here forse, at use, this reage wherell;\n",
      "And there so t \n",
      "\n",
      "[1m 54s (1200 60%) 1.5400]\n",
      "Whave chardy and 'teen have not thine and deart it one\n",
      "I'll me you not loud but vering make you so tak \n",
      "\n",
      "[2m 5s (1300 65%) 1.5484]\n",
      "Wher-falin, all the with the this would heart:\n",
      "Bears of we him, roull\n",
      "Un love to vonauted with sit poo \n",
      "\n",
      "[2m 15s (1400 70%) 1.8975]\n",
      "Whirst of the with thuns\n",
      "As men the beak me fraident, but thee the me to for prider:\n",
      "And strut youts a \n",
      "\n",
      "[2m 25s (1500 75%) 2.0092]\n",
      "What her colsting of aut tell on Duke\n",
      "When shalf see the till fing hard the and these, a for Morteous? \n",
      "\n",
      "[2m 35s (1600 80%) 1.9845]\n",
      "What her tane gentrearge heseed soul thy bleed.\n",
      "\n",
      "DUKE VINCENTIO:\n",
      "Why they prows of Gell here with up p \n",
      "\n",
      "[2m 45s (1700 85%) 1.6289]\n",
      "Whand,\n",
      "Scan the ford, but the anstles subject:\n",
      "Shall the mes. Got and the call;\n",
      "And of the cart corse: \n",
      "\n",
      "[2m 54s (1800 90%) 1.6927]\n",
      "When the know o' matield and not counder's crown:\n",
      "What heres leaved one the knows the manust now s wan \n",
      "\n",
      "[3m 4s (1900 95%) 1.9071]\n",
      "What brother the gradieds: I beward,\n",
      "The so vister go fouth you sway.\n",
      "\n",
      "DUKE VINCENTIO:\n",
      "I good more agi \n",
      "\n",
      "[3m 15s (2000 100%) 1.6370]\n",
      "Wheremus, suc!\n",
      "\n",
      "LUCIO:\n",
      "Lost, look'd lif, the torms to shall bear's\n",
      "For that should intreak ander disce \n",
      "\n"
     ]
    }
   ],
   "source": [
    "n_epochs = 2000\n",
    "print_every = 100\n",
    "plot_every = 10\n",
    "hidden_size = 100\n",
    "n_layers = 1\n",
    "lr = 0.005\n",
    "\n",
    "decoder = RNN(n_characters, hidden_size, n_characters, n_layers)\n",
    "decoder_optimizer = torch.optim.Adam(decoder.parameters(), lr=lr)\n",
    "criterion = nn.CrossEntropyLoss()\n",
    "\n",
    "start = time.time()\n",
    "all_losses = []\n",
    "loss_avg = 0\n",
    "\n",
    "for epoch in range(1, n_epochs + 1):\n",
    "    loss = train(*random_training_set())       \n",
    "    loss_avg += loss\n",
    "\n",
    "    if epoch % print_every == 0:\n",
    "        print('[%s (%d %d%%) %.4f]' % (time_since(start), epoch, epoch / n_epochs * 100, loss))\n",
    "        print(evaluate('Wh', 100), '\\n')\n",
    "\n",
    "    if epoch % plot_every == 0:\n",
    "        all_losses.append(loss_avg / plot_every)\n",
    "        loss_avg = 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 损失变化图\n",
    "\n",
    "查看损失在训练过程中的变化有助于我们了解学习的过程。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7f55908347b8>]"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xd8XNWd9/HPmabee7Usd9nGFTfAsamGpQRIqIEkQICQ7EI2CZtAQspudpMnC3lgSRZIyEMJxdRgCN0Yg8FNLrItN8m2LMnqktWtMprz/HHvjFVmZNmWNJrx7/166eXRzJV0dDX+zpnfPUVprRFCCBFcLP5ugBBCiOEn4S6EEEFIwl0IIYKQhLsQQgQhCXchhAhCEu5CCBGEJNyFECIISbgLIUQQknAXQoggZPPXD05MTNQ5OTn++vFCCBGQtmzZUqe1TjrRcX4L95ycHPLz8/3144UQIiAppQ4P5TgpywghRBCScBdCiCAk4S6EEEFIwl0IIYKQhLsQQgQhCXchhAhCEu5CCBGEAi7c91W18PCH+6hv7fR3U4QQYswKuHAvrmnlfz4ppq61y99NEUKIMSvgwt1mVQB097j83BIhhBi7Ai7c7Wa4O13azy0RQoixK+DC3WYxmuyUnrsQQvgUcOFutxpN7pJwF0IInwIw3M2yTI+UZYQQwpeAC3eb2XN3uqTnLoQQvgReuFvco2Wk5y6EEL4EXLi7a+5SlhFCCN8CLtxtnqGQUpYRQghfAi7c7eZQSCnLCCGEbwEX7p6euwyFFEIInwI23LtlhqoQQvgUcOHuMC+odjul5y6EEL4EXLjLOHchhDixwAt3GecuhBAnFHDhLuPchRDixE4Y7kqpUKXUJqVUgVKqUCn1Ky/HfEspVauU2m5+3DEyzQWrRaGUlGWEEGIwtiEc0wmcr7VuVUrZgXVKqfe01hv6HbdSa/394W/iQHaLRcoyQggxiBOGu9ZaA63mp3bzw6/JarMqGecuhBCDGFLNXSllVUptB2qAj7TWG70cdq1SaodS6jWlVNawtrIfm0XJTkxCCDGIIYW71rpHaz0byAQWKKVm9DvkbSBHa30W8BHwrLfvo5S6UymVr5TKr62tPeVG260W2UNVCCEGcVKjZbTWjcAaYEW/++u11p3mp38B5vn4+qe01vO11vOTkpJOpb2AEe4yWkYIIXwbymiZJKVUrHk7DLgI2NvvmLRen14J7BnORvZnsyrpuQshxCCGMlomDXhWKWXFeDF4RWv9jlLq10C+1noV8C9KqSsBJ9AAfGukGgxmWUZq7kII4dNQRsvsAOZ4uf+hXrd/Cvx0eJvmm80io2WEEGIwATdDFYz1ZWScuxBC+BaQ4W63KpmhKoQQgwjIcDfKMtJzF0IIXwIz3GWcuxBCDCogw91htcgMVSGEGERAhrusLSOEEIMLzHCXVSGFEGJQARnudpmhKoQQgwrIcLdJzV0IIQYVkOFut0jPXQghBhOQ4W5cUJWeuxBC+BKg4W6RGapCCDGIgAx3oywjPXchhPAlMMPdapFx7kIIMYiADHebrOcuhBCDCshwt8sMVSGEGFRAhrvNYsGloUd670II4VVghrtVAchYdyGE8CEgw91uhrvMUhVCCO8CMtxtFqPZUncXQgjvAjLc7Z6yjPTchRDCm4AMd5vV7LnLLFUhhPAqIMPd7g536bkLIYRXARruMlpGCCEGE5Dh7rmgKqNlhBDCq8AMd+m5CyHEoAIy3GW0jBBCDC4gw13GuQshxOACM9yl5y6EEIMKyHC3yzh3IYQYVECGu81iri0jPXchhPAqIMPd3XOX0TJCCOFdQIe7jHMXQgjvAjLcZZy7EEIMLiDD3W6RtWWEEGIwARnu0nMXQojBBXa4S81dCCG8Cshwt8sMVSGEGFRAhru75y41dyGE8O6E4a6UClVKbVJKFSilCpVSv/JyTIhSaqVSqlgptVEplTMSjXXzjHOXGapCCOHVUHruncD5WutZwGxghVJqUb9jbgeOaq0nAn8Afje8zexLdmISQojBnTDctaHV/NRufvRP1auAZ83brwEXKKXUsLWyH6tFoZTU3IUQwpch1dyVUlal1HagBvhIa72x3yEZQBmA1toJNAEJw9nQ/uwWi4yWEUIIH4YU7lrrHq31bCATWKCUmnEqP0wpdadSKl8plV9bW3sq38LDZlXScxdCCB9OarSM1roRWAOs6PfQESALQCllA2KAei9f/5TWer7Wen5SUtKptdhksyhZz10IIXwYymiZJKVUrHk7DLgI2NvvsFXAN83bXwM+0VqPaPLarRaZoSqEED7YhnBMGvCsUsqK8WLwitb6HaXUr4F8rfUq4GngeaVUMdAA3DBiLTYZZRnpuQshhDcnDHet9Q5gjpf7H+p1uwP4+vA2bXA2i0XGuQshhA8BOUMVwC49dyGE8CmAw90ie6gKIYQPARvuNquFLqf03IUQwpuADfeYMBvNx7r93QwhhBiTAjbcU6JDqW7p8HczhBBiTArocK9q6mCEh9MLIURACthwT44KodPpovmY099NEUKIMSdgwz0lOhRASjNCCOFF4Id7s4S7EEL0F7DhnmqGe1WThLsQQvQXsOGeHB0CQE1Lp59bIoQQY0/Ahnuo3UpMmF3KMkII4UXAhjtASnSIhLsQQngR4OEeSlWzlGWEEKK/gA735KhQaqTnLoQQAwR0uKfGhFDT0olLNsoWQog+AjrcU6JD6XFp6tu6/N0UIYQYUwI63JOjZCKTEEJ4E9Dhnh5rhPuRxmN+bokQQowtAR3umXHhAJQflXAXQojeAjrc48LthDusHJFwF0KIPgI63JVSZMaFUX603d9NEUKIMSWgwx2M0oyUZYQQoq8gCHfpuQshRH9BEe7NHU6aZLNsIYTwCIJwN0bMyEVVIYQ4LgjCPQxASjNCCNFLEIS72XOXiUxCCOER8OEeF24nzG6VETNCCNFLwIe7jHUXQoiBAj7cAbLjwzlcL+EuhBBuQRHuE5IjOVTXRo+s6y6EEECwhHtSBJ1OFxVyUVUIIYCgCfdIAIprW/3cEiGEGBuCItxzzXA/UNNKc0c39a2yabYQ4swWFOEeH+EgLtzOgdo27n1pG3c8l+/vJgkhhF/Z/N2A4TIhKZL8kgYO1LYSERI0v5YQQpySoOi5gxHuRTWtuDS0dDhp7pCFxIQQZ67gCffkiD6fy8gZIcSZ7IThrpTKUkqtUUrtVkoVKqXu9XLMMqVUk1Jqu/nx0Mg017fcROOi6vIpSYCEuxDizDaU4rQT+KHWeqtSKgrYopT6SGu9u99xn2utLx/+Jg7Nwtx4rp2bybfPyWHNvlpZAlgIcUY7YbhrrSuBSvN2i1JqD5AB9A93v4oKtfPwdbNwuTQOq4UjjR3+bpIQQvjNSdXclVI5wBxgo5eHFyulCpRS7ymlpg9D206JxaJIiw2VJYCFEGe0IY8ZVEpFAq8D92mtm/s9vBUYp7VuVUpdBvwdmOTle9wJ3AmQnZ19yo0+kfSYMKm5CyHOaEPquSul7BjB/oLW+o3+j2utm7XWrebtdwG7UirRy3FPaa3na63nJyUlnWbTfcuIk3AXQpzZhjJaRgFPA3u01o/4OCbVPA6l1ALz+9YPZ0NPRnpsGNXNHXT3uPzVBCGE8KuhlGXOAW4Bdiqltpv3PQBkA2itnwC+BnxXKeUEjgE3aK39tv5uZmwYLg1VTR1kxYf7qxlCCOE3Qxktsw5QJzjmceDx4WrU6UqPNTbNPtJ4TMJdCHFGCpoZqr2NSzACfV9Vi59bIoQQ/hGU4Z4VH874xAhW763xd1OEEMIvgjLcAS6YmsyGA/W0djr93RQhhBh1QRvuF+al0NXjYl1Rrb+bIoQQoy5ow33+uDhiwux8tFtKM0KIM0/QhrvNamH5lCQ+2VuNU8a7CyHOMEEb7gArZqRxtL2b9Qf9Np9KCCH8IqjDfdmUJCIcVt7dWenvpgghxKgK6nAPtVu5YFoK7++qktKMEOKMEtThDnDZTKM088bWI/5uihBCjJqgD/dlU5I4KzOG+1/fwW/+YewvUtF4jPekVCOECGJBH+6hdiuv3b2Ey89K4+l1hzjW1cNTnx3kuy9spaZZdmsSQgSnoA93AIfNwuVnpePSsLeqmZ1HmgD4vKjOzy0TQoiRcUaEO8D09GgAdh5pYneFsZHU2v0ye1UIEZzOmHDPjAsjJszOOzsqOdbdQ2SIjXXFdbhcflt2XgghRswZE+5KKfLSotl0qAGAmxdl09DWxa4Ko0Sz4WC9bKothAgaZ0y4w/HSjMNm4bZzxgPwYWE1FY3H+MZfNvLYx0X+bJ4QQgyboWyzFzSmZxjhPi01ipToUC7OS+GZL0soP9qO06UpqpHNPYQQweEM67nHGP9mGP/ev2Iqx7p7+Pv2CgAO1Lbhx61fhRBi2JxR4Z6bGMHFeSlccVY6ABOTI7nh7CyUghsXZNN0rJv6ti4/t1IIIU7fGVWWsVktPHXr/D73/fzyPG5cYFxcfWlTKcU1rSRGhviphUIIMTzOqJ67N6F2KzMyYpiQHAnAgdpWP7dICCFO3xkf7m5p0aGE2a0cqGnzeUx1cweVTceHS2qteW1LOce6ekajiUIIMWQS7iaLRTEhOWLQnvtdz2/hhqc20G0uH1xY0cyPXi3glfyy0WqmEEIMiYR7LxOSIimsaObrT3zJH9cUA7CqoII1e2uobu5ge1kjh+vbeWNrOQAl9UYvP//wUb+1WQghvJFw72VCUiR1rZ1sLjnKs1+WcKyrhwfe2MmPXyvg/V1VAKTHhPLY6mK6nC4O17cDsKWkwZ/NFkKIASTce7lsZiqXTE/hXy6YRE1LJ4+uLqK100ldaxf//cE+MmLD+PevzuBI4zHW7q+lrMEI94qmDioaj3GwtpUup+z4JITwPwn3XiYmR/HkLfO5/Zzx2CyKpz47QHyEgykpUbR0OrlgWjJLJiRiUbDrSBOH69uJCjVGkz6+ppjzH17LzX/ZQIOMlRdC+JmEuxcx4XYWT0jApeHSGal8Z2kuABflpRDmsDI+MYLCimZKG9pZNiWZcIeVFzeWkhwVQkF5Ezf9eYPMdBVC+JWEuw8rZqQCcMWsdK6dm8Gb9yzh3ImJgLGMQUF5IxVNx8hNjGB2ViwAf7h+Nv9x1Qz2VrWwtfQoGw/W8+u3dw8p6HeUN1JULWvbCCGGxxk1Q/VkXD8/i/EJESzKTQBgTnac57Hp6dGsKjDWoxmXEM7C3HgumZ7KORMTae108tCqXby57QibDjWwv7qVmxZmM9GcJOXLv75SQEyYnde/u2TkfikhxBlDwt0Hm9XCErOn3p97ATIwwn3euHiWTDCOjQyxceG0FF7aVEaPuRHI2v21g4Z7l9PFobo2FNDe5STcIX8WIcTpkbLMKcgz14UHyI6PGPD4VbMz6HFpxidGkJsUccLt/Erq2+hxaZwuzRYZMy+EGAYS7qcgPsJBWkwo4Q4riZGOAY9/ZXISi3MTePCyaSybnMzGg/V0dPteoqCo+vis2I0HfY+Zr2npoKRu4PIIL28qZeYvP+DxT4rodMpSCEIICfdTNj8nnry0aJRSAx5z2Cy8dOciLsxL4StTkuh0ulh/sJ7uHhfPfHGIQnNrP7fimlaUgqmpUWw4WM9n+2sH9PY/KKziwofXct2T6wdcoN1W2khbp5P//nA/T649OPy/rBAi4Ehx9xT99pqZOIewufbC8fFEh9q47+XtZMaFUVjRzCXTU3jyluNLDxfXtpIZF8ayKck8+dkBbv3rJgDuu3AS914wiYLyJu56fgsxYXZqWjo5UNvWp4Zf0XSMmZmxdDldUtYRQgDScz9lESE2YsLsJzwu1G7lte8u4azMGA7XtzM1NYoth4+itWbl5lI+L6qlqLqFiUmRLJ+ShNZw44Isrpmbwf/9uIin1x3isdVFxIbbeeGOhQCeTb7djjQeIyM2lOnp0RRWNI/I7yuECCzScx8Fk1OieP72hTh7XKzML+PBN3dRWNHMz/9eSGSojdZOJ0snJ7EwN4H8n11IYmQIWmtaOpz87v29dPdofnTxZKanR5MYGcLmkgZuWpgNGMsOVzQe4/wpyaTHhvHalnJqWjpIjgplzb4afvRKAe/ft5SkKNmARIgzifTcR5HNamH+uHgA/s8H++jqcdHQ1kWX08XEJKPM4t4FSinF7649i9hwB9GhNm5dkoNSigXj4/r03I+2d9PR7SI9Nswzime32Xv/aHc19W1dvLuzcjR/TSHEGCDhPsomJUcSFWrjs/21JEeFcMPZWQBMTBk4Dj4+wsFrdy/mpTsXER1qlIDOzonnSOMxjjQam4ZUmP+mx4YxLc0M90oj3LeUGPX3f+yQcBfiTHPCcFdKZSml1iildiulCpVS93o5RimlHlNKFSuldiil5o5McwOfxaKYa852XTEjlQf/aRr/dc1MZmfGej1+XEJEn0lTC8YbPf8n1x7A2ePyhHx6bCgxYXYy48LYXdFMU3s3+6pbSIhwsPlwA1VNHSP8mwkhxpKh9NydwA+11nnAIuB7Sqm8fsdcCkwyP+4E/ndYWxlk5o8zwv3SGWlEhdq5cUE2FsvAIZXe5KVFc9PCbJ5bf5jbns3v03N3P767spktpUbp5gcXTUZreGdHxWm3u7alk0/31Zz29zkZje1dfLy7elR/phDB4IThrrWu1FpvNW+3AHuAjH6HXQU8pw0bgFilVNqwtzZI3Lgwm4cuz2Oh2Qs/GUop/vPqmfzgwsl8tr+Wz4vqcNgsJEQYk6mmp8dwqK6N59cfxmZRXDs3k7nZsTzy0X62lp7eMMnHVhdx2zObaet0ntb3ORnPrT/MHc/lc/AUNy4vrmnBNYQhq0IEm5OquSulcoA5wMZ+D2UAvTcSLWfgCwBKqTuVUvlKqfza2sGn5AezxMgQbjt3/JB7695cd3YmAGv21ZARG+aZTHXTwmyy48NZs6+W6RkxhDmsPPGNeSRFhfCtv24aUnmmtdPp9bgNB+txaSiqObWgPRV7q4zrB5/uO/nnS3FNKxf94TPeM3fREuJMMuRwV0pFAq8D92mtT2kwtdb6Ka31fK31/KSkpFP5FsKUFhPGrMwYtDbq7W5JUSG8cMdCJiZHcvlM481TcnQoz3x7Ae1dPTyx9gDdPS6+PFDnWdisN601dz2fz/VPre9zf11rpyfU91eN7NLEzh4X9a2dAOwzf9aaUygHbS5pQGvY1W9GsBBngiGNc1dK2TGC/QWt9RteDjkCZPX6PNO8T4ygi6enUlDeRHpMWJ/7M+PC+egHS/vcNz4xgmvnZvLiplLKGtpZvbeGe5ZN4P4VU/sc99Huar4orgegqb2bmHBjlE7vNW/29Vp3vqqpg5gwO2EOa5/v85/v7iEhwsFdX5nAH9cU83ZBBUlRIfz22rPIiO3b3v4eXV3EM1+WsO7+8ympb8dhtbDxUMMJV8ysae7ApSE1xnix22rO1u29ds/JeH59CWEOG9fOzeizzESX04VLa0LtVt9fLISfDWW0jAKeBvZorR/xcdgq4FZz1MwioElrLePvRtgl01MAyIgbGJZKqQHr3tyzfAI9Ls3qvTXkpUXzp08P8Oa2clo7nfxg5XYuemQtD7y5ixCb8bTYX3M8xDccrCfCYWVqahT7zXDfeLCeZf+9ht9/sK/Pz2nvcvLMFyW8uKkUrTXPrS+hpcPJF8V1rNxUOujv5Oxx8fLmMlo6nLy0uZQel+aauRl0OV2sP1A/4PgvD9SxvawRgLv/toVv/b9NnrV3tpn3F9ec/DuNutZOfvX2bn70agEPvLmrT93+3pe3cf2T6+lxaRraujwXtUfSmr01PPjmTl7ZXCaLw4khGUpZ5hzgFuB8pdR28+MypdTdSqm7zWPeBQ4CxcCfgXtGprmit4nJUTx6w2xuWpA9pOPHJUTwiyvy+M3VM3jjniXMzorlBysLWPSfq1lVUEFKdCghNgu/uXomcLwkAka4z8+JZ3p6DPuqWiisaOK2ZzbT0e0aUPZYf6Cerh4Xh+vbKaxoprq5kzuX5rIoN4G3d1TS49L8fdsR6szSS2+fF9VR22Lc/8wXJQB8Y9E4IhxW3u9XO+/o7uG7f9vKD1/ZTk1LB1tLG9lb1cKuI800HeumuKaVqBAbhxvaB12V05u3CypwujRXzErnpU2lbDh4/IVlT2UzBeVN/HFNMVf9cR23PbP5pL73qXjmyxJe2FjK/a/vkHkLYkhOWJbRWq8DBr3yp42u0veGq1Fi6K6aPeC69aBuXZzjub3yrkW8vuUI7+yo4J5lEzl3krHhiNaaX60q9PTQ39tZSVFNK9efnYVLa17fWs5P39hJmMPKwtwECsweslvv+vhf1x0CYG52HCE2Cz95Yyf3vryNd3ZUkhjp4JHrZrN0chIvbyrltS3ldLs08REOJiZHsulQA3arYnJKFFfOTufNbUd48J+m8cbWI2THh9Pa6aTpWDdNx7r505oDACgFr28t5/ypyQBcMTudFzeWcqC2tc98ATCGdu6pbGbp5IHXf97YeoQZGdH87tqZfLCrijX7algyMRGXS1PRaFxsfuSj/Z6f2drpJDJk5FbzONrexbkTE9l4qJ79p1hmEmcWmaF6BguxWblpYTYvfmeRJ9jBKOlMTo1iX1UL+6pa+OGrBczJjuWWxeOYnBIFwI7yJu5aOoElExKob+vyXADVWvPpvlrOnZiIzaJYVVBBqN3C1LQoVsxIxW5VvLOjkuVTkkiICOE7z+WzrfQov3l3DwXljRSUNXLV7HRPOOcmRuKwWbh1cQ4d3S6+9+JWfv3Obu55cSuPrykmLSYUm0Xx7PoS0mJCuXRGKqsKKvigsAql4Nq5xqii4n4jfJw9Lu58Pp9v/r9N1LQcHxn0P6uL+OeXtrHzSBPXzMkk3GFjYW48a8zROrWtnXT1uLhl0TgmJEVw88JstDZ68yOpoa2L5KgQsuPDT3lYqDizSLgLryanRLGvuoWfv7WLMLsxnDLEZmVKqhHuCREObl50fG/Y4ppWNh1q4OEP91N+9BgrZqQyLS0ap0tzVkYsdquF2HAHF05LISs+jP97wxyeu30BDpuFG57aQGunk7e+dy5/u30hP7x4CueZLzaTzZ83LS2aBePj+aK4nrMyY4gNs1Nc08o3Fo3jvEmJaA3LpybztXmZNLR18cLGUmakxzAzIwarRQ24qPrE2gNsK21Ea1i9x3in0d3j4tHVRXy8u5qU6BCunJ0OGJuvFNe0UtbQTvnRdgDOn5bM6h8u418umATAriMDR+R097i4+/ktfUo6p+poWxdxEQ5ykyI56GXDFiH6k3AXXk1JiaSxvZtNhxr4/vkTSYk2RqCkRoeyICeeH18yhXCHzRPueyqbuf3ZzTy+ppiECAcX5aUwN9tYUmHOuONLK/zh+tm8d+9SYsLspESH8rN/mkan08U1czLJS4/m3EmJRIbYmJYazdLJSZ6LxgD3XTCJvLRoHr9xLn+8eS5LJydx/dlZntLUhdOSWT4lmWe+fTYv3rGQ524zXjxyEsLZeaSJneVNdPe4aOt08tjqYv5pZhqZcWF8ZM6ALW1ox+nS/MdXZ7DxgQs9i7gtm2K8i/h0fy3lR42Lp1nmRezkqBASI0PYdcTouTd3dHPxH9ayek81n+yt4f3CqlMao99bR3cPbV09xEc4mJAUyeH6Npw9Lq/H9t/IRfjm7HFx2aOf8/6u4LyGIUv+Cq/cPeaU6BBu7HXBVinFK3cv9nyeHhNGuMPKc+sP09Lh5C+3zufCPCOQ546L49n1hz1r6QADhg9eNz+LmDD7gM3ILRbFc7ct6HPfkomJvHvveQBkJ4R7Hr9yVjrxEQ7Om5SIUsoTxm5TUqN4d2cVa/fX8l/XzCQvLZquHhdXzEonOTqEFzaW0tbp5GCt0SPOTeq7L+6EpAiy4sNYu6+WOeYLlnu5B6UUMzOiPT33Dwur2V/dym/f20tWfDhAn7JPb5sONfDHNcU88Y15A4aS9tbY3g1AXLiDpKgQuns05UePkZPYt50vbizlfz4pYu2Pl2O3Kjq6XYN+3zNdbWsnuyub+byojhUzgm9CvfTchVfT02KIDLHxrxdNHnQ8t8WimJhslAqiQ219Lk5eMj2VX1yR56mfe6OUYsWMNM+ql6fCYlEsnZzkdctDgB9cOJmfX55HTJidgrJGz2SsSSmRXJSXQpfTxedFtRwwa9m5SX1X6FRKsTg3gc0lDZQ1tJMQ4egz3n5GRgxFNS0c6+rhHzsqsFkURTWtfLLXKPe4R//01tzRzb0vb2Pt/lo2l/jeNxegvs34+vgIOxPMF56DdQPr7vklDVQ2dbC19Cgvby5jwX9+THNH96Df+1S8sbWcq//0RcC/S6hpNs7roSAtc0m4C69iwu1s/flFXH/2iYdZuteivygvFYft+FMq1G7l2+eMx27179NsUkoUt587nhkZxqJqRdUtOKwWxsWHsyDH2Abxk701HKhpJTEyxOsOWwvGJ9B0rJu1+2sHzCuYnh6DS8O64jo+L6rjW0tyPL3/ScmRVDcP7Ln/atVualo6sVrUgJ21+jvadrznnptonOuDtW10dPf0CdhD9UZIrSuq4/Ut5bR0OMk/wQuHNx3dPfzryu2UNbQPeKzHpfnDx/vZVtpIQ1vXSX/v0dJivngONgfB/XcZrXAvqm7hL5+P3h7HEu7Cp95BPRj3WvSXnzW239pOS41mb1ULe6payE2KwGa1YLNaWDIhkXVFdRyobfX0jPtbkGMs8lbZ1EFmv3CfnxNHVIiNu/+2BadLc+XsdP7r6pncv2IKC3PjqenXc/+wsIrXt5Zzz7IJzEiPZtMJArih3QjRhEgHcREO4iMcvF1Qwexff8gtT2+iyByyerjeCON3dlSwxVwkrvfM4qEqKGvkjW1H+HjPwNU4P9lbQ1nDMc+5GKu2HD7KW9srePbLEp/HVJt/l8qmDtq7Rn4xvBc2lvIf/9jj9UVzJEi4i9N21ewMvrtsQp/hlGNRXno0XU4XGw7UM8kc0glw7qREKpo62FHeNKAk45YVH0aqeVE5My68z2OJkSG89f1zmJ4ezbS0aGZmxLAwN4F7lk0kOSqUxvZuz6zSutZOHnhzJ9PTo/nn8ydxdk4828saB53enIQdAAATV0lEQVR1etTsIceFGyt/5iZGUFDeRGJkCDvKG7nxzxtobO+ioa2L2HA7JfXtaG1c/N5wgncF3hSb5Sn3i0Vvz3x5CLvVKH8NZRG6j3ZX8+WBupNuw1DVNHfw/PoSupzGBWb3ZLVSM0Bf33qEbvPic1uns8/Eudpe76hK6kYmcHtc2tMmd9lv/TCMnhoKCXdx2jJiw/i3FVP9Xn45EfdOVV09LiYnHw/xpZOM6wROl/bZcze2ODR6797WxslNiuSt753D298/p0/tPyXaGHFT29JJcU0rX39iPc0dTh6+bhYOm4Wzx8fT5XSxo9z34mYNbV0ohadcNCsrlpToEF76ziJ+eeV06lq7+LDQ6GVfN99Y4ik3MYKvzctk15EmWk9yiWb3nIDSfj3MyqZjfFFc77nAXuml3FRQ1sjtz2zmz58dpK61k39+aSsPvrnLUz7SWrPxYP2wLMNcXNPC1X/6kp+/VcjLm0tZvaeaGb/4gL1VzZSaL0x1rZ2e0Uq/e38v1z15fEG86ubjQT9cpZn8kgbPiwkYy2Rf+MhatNae8+ptGY2RMLb/NwoxjCYkReIwX4Am9drWMDshnGxzZMuEZO89d4CzzXDvX5ZxU0ph6/cClxxl9Parmjq46c8baOno5oU7FjI11XihOdss9zz84T4eemsXHd091LV28v0Xt/LCxsO0dTo52t5FTJjd870fuGwan/5oOVnx4cwzN355bWs5AF+dnUFyVAjXzM1gYW48PS7NlsMD1/FfudmYEeyNO4QO1/cNPHdI3rQwG5tFUdmvnr1mbw1X/fELPt1fy2/f38tPXt9BR7eLQ3Vt7DWXssg/fJTrn9rAP05iX9/imtYBewgcqG3l+ic30Ol0MTU1isc/KeYXqwpxujRbDzdS2tBObmIESVEhvJJvrEa+t7LFc60CoLqlw/NifsjLBeqhKGto56xffsDW0qPsLG/ia0+s563txzfGeX9XFeVHj1FY0ewpY315oG5ULkZLuIszhsNm8YzL712WATwlpQmJvsP9shmpXDM3wxPyQ5EUZfTcNx5qoKalk39bMdUT6GDsk7sgJ578kqM8t/4w6w/U80FhFe/sqOTBN3dx78vbaGjrIt4syQBYLcozxDE7PpzESAebDjWglDGM87P7l3PPsonMGxeHzaIGTKJqOtbNQ28V8qNXC3g1v4z+3OFedvRYnx72mr3G3gFTUqJIiQ6lqqmDXUea+PGrBXT3uHhz2xESIx18dv9y4iMcfLynhqWTk7AoPJu0u0cGDXUJ51fzy7j4D2u54akNnpE/20qPcstfNqIUvHKX8e6lpqWT8qPHsFkU+6tbjHBPimD5lCS2mdcfyswJaO65CtXNnYxPjCAtJvSUJ4blH26gucPJ37cd4SPzGsWOcmM5jprmDs8Kqm9sNRbJXTo5iermzlG5iCvhLs4o09OjcdiMkTK9fXtJDnd9JddnrxwgITKER66bfVLDNpPNssxq8z/+zMyYAcesvGsRBb+4GKtFsbmkgS0lR0mMdHDzwmzWFddR09JJXIRjwNeB8W7BPY8gPSaMULuVULsVi0UR7rAxd1wca/b2DdK3Cyo8Pd6fvLHTE35grJFT2dRBRmwYXU4XVWbppdPZwxfFdSyfagw5TYsJpbKpg9e3lvPqlnI+LKzm0301LJuSTEZsGL+9ZiZZ8WH84oo8Fo5P4B87K9Fas63UCL7P9tcNWpqpb+3kl6sK+fFrO5iZEcOeymauevwLLnv0c67+05d0Ol08d9tCcpMiWZSbwNfnZXLHueOZkRFjlGUa2smKDyc3KZK61i5qWzo9v4v7gmZtSwdJUaGMT4xgT2UL//vpgRMOS+1vb6UR3h8WVnv+xrsrjAlt64qPX2t4a7sR7rcsGgfAl6NQmpFwF2eUey+cxNPfnD+gfDIpJYqfXjrttHbH8iYhIgSrRbGtrJEQm8UzbLQ3pRQRITZmpEeTf/go+YePMm9cHItyE+jodrG9rNFzMdWb+TlGuI9PHHi94KJpKeytavEsmwBGb3hqahSv3r2Y+AgHv3p7t6dMcMDstbvnJhyubye/pIGVm8to6+phuTlBLDUmlKrmDs/krX9/ZzfNHU4uML/ugmkpfPbj5UxIiuSys9I4WNvGnsoWtpU2EhVqo661kz3mLlvv76riey9uZeXmUtq7nOytaub8h9fy3PoSbl6YzSt3L+bxm+YSG24nIdLBA5dN5bP7l5OXHu35nX7/9Vn87PI8pqZGsa20kfauHrLjw8k1z8nnRbW4KyGlDe1097ioa+0iJTrEDPdmfvf+Xq5/cj2PrS7qUzbZXNIwYG0iN3e5qaq5g8KKZkLtFvZUNuNyadYV1XnemdW3dWG1KL4yOYm8tOiTXqX0VEi4izNKZlw4500avV3ArBZFYqQDrY0Luv1fVHqbnxPPttKjlDa0M39cvGc2bJfTRXyE73cL7rr7uITwAY+5Zwu7189xL1d83fwsokLt3H/JFLaXNbKqwKgTuyd4XTDNCOkPCqv42hPreeitQkLtxrBRgLSYUCoajVpyiM1CVXMHdqsasAAdwOUz0wixWfjt+3upa+3k2+eMB2DtfqOG//yGEv6xo5J/e30nFz3yGbc+vYkwu5UP7lvKb66eSYjNyooZqbx5zzk8f/tC7lw6gQgfK3BOSY2i0xw5My4h3DP6yf2zwAh398SylOhQrp6TwVWz01l55yKumJXOIx/t521zWWWtNXc/v4W7/7bF685le6uaOX9qMlazU3DzwnG0dfVwqL6NdcV1LJmQ4Fl+Y1xCOA6bhXfvPY87zsv12v7hJOEuxAhzX1SdkRE96HHzx8XR3WMEyLycODJiwzw1+/iIEJ9fNz09hvGJESzKTRjw2PjECHKTIjxj1h9fU0yEw8rVc4z1eK6dm8n09GgeNXurxTWt2K2KRbkJ2CyKFzYeJsRm4clb5vHidxZ5av2pMWF0Ol20d/Vw91cmoBQsHJ9AlJeSVVyEg6/OzuAzM2AvzkshLy2atftq0VpTWNHM9fOzePnORUSEWOl0unj2tgUDrosMxZReX5Mdb1wot1qU52fHhdspbWj3zD1IiQ5hfk48j94wh4W5CTxy3WxmZcXyy1WF1LcatfH6ti6Ka1p5Z0dFn5/V0NZFdXMni3MTWJQbT0ZsmOe8PrX2IDUtnSyfkszsTCPcJ/gYZjtSZG0ZIUZYshnQMzMG1tt7m2eWV0JsFmakx6CUYk5WLB/urh605x5qt7LmR8t8Pn7RtBT++sUh/rbhMP/YUcm/nD/RU8O3WBS3Lh7Hv72+k4LyJtYfrGdichShdisZcWEcrm/n2rkZXDI9tc/3TIs5vm/vpTNTSYoK8Qw19eabS3JYmV9GmN3YzevcSYk880UJB2rbaGzvZkZGNItyE3jv3qUc6+455bXx3auWgvEuzWGzkBUXRkl9OyE2C3Oz4yhraPfMTnW/8LpZLYrff+0sLn9sHY+uLvL8zRIjQ3h0dRGXn5Xu6aW7N2+fkmrsN9De1UNGbBh2q2JlfhlpMaFcPivNM5N34iAjsUaC9NyFGGHui6r9NwsZcJx5cW9WVqxndvAc82LpYDX3E/nGonGkxYTxs7/vIibMzh1L+5YELjXLJj/7+04Kyhq5eaExjt09PPTmRQOXoHCHe6jduI7wjUXjPOUhb/LSo1k2JYlzJiZgs1pYnJtAV4+Lv204bD5unBurRZ3WpicJkcYqnanRoZ41kdylmaz4cLITwintHe7RA98RTU6J4pIZqbxdUMHGQw1Eh9r46aVTOVjb1mdpZ/dOZVPTjNFD4xMjcNgsTEo2XmDu/soEQmxW0mLC+O+vz/JcTB0t0nMXYoRNSYkiIcLh2ehkMP/7jbmE2I4v1OaeOJV+gk3FB5MVH867957H/6wuYlZW7IDRPtGhdi7KS+GdHZWkRofy9fnGBicXTkshwmFjTlbsgO+ZZm7KfqLrCL39+db5ni3d5ufEYbUoXskvw6JgWtrJl2B8OTsnrs9EotzECD7heJmmvauH7aWNWC2KBB/lritnpfN2QQVvbT/CORMTOcsc5XSoro24cAe3PbuZ9k4nCREOkiL7fo8F4+NpbO/i+rOzPPd9bV7msP1+QyXhLsQIu3VxDtednTWktXrck5vc5o2L4+3vn3vCev2JRIbY+Oll03w+fu28TN7ZUcl3l03wvLh8c0kO31yS4/X4pKgQHDYLszIHBr8vvWcwR4XamZERQ0FZIxOTI/ussnm6/nD97D6fjzcnKrnDHeCNbUdYPiXJU2Lpb+nkRKJDbTR3OJmXHUdWfDhKGeHe0d1DcU0rCREOlk1JHrAa6QOXTeOHFw++mupokHAXYoS5x5yfKm9j44fbsslJvHr3YuZl+y6t9Ga1KJ6/bcGgM3pPZFFuPAVljcxIP70Xrv76h6p7Jc3MuDDPiKKU6BB+//VZPr+He4TOK/nlzBsXR6jdSnpMGCX1bbR0OAmzW9n84IVeh846bJYhL7o3kiTchRAopfrMnB2KhV5G55yMxbkJPLn24AmvRZyumZkxLJmQwFcmJ5GTEMENZ2dxw4Jsz05bvtxxXi7tXT3MHXd8HkFJXRsNbV1MTI4c9jkRw03CXQjhF4snJHD7ueM9e9WOlMgQGy9+Z5Hn899ee9aQvm5yShSP3zTX83lOYjirtlcQ5rByzsSxvQIqSLgLIfwkxGbl55fn+bsZQ5aTEEFzh5PmDueQLo77m/8LQ0IIEQB6L+8wOWV0x6yfCgl3IYQYgt4bkrvHso9lUpYRQoghyIoLx6KM0TjeNmwZayTchRBiCBw2C5lx4cSF28f8SBmQcBdCiCH74cWTiRjGCVcjKTBaKYQQY8BVszP83YQhkwuqQggRhCTchRAiCEm4CyFEEJJwF0KIICThLoQQQUjCXQghgpCEuxBCBCEJdyGECEJKa+2fH6xULXD4FL88EagbxuYMp7HaNmnXyRmr7YKx2zZp18k51XaN01onneggv4X76VBK5Wut5/u7Hd6M1bZJu07OWG0XjN22SbtOzki3S8oyQggRhCTchRAiCAVquD/l7wYMYqy2Tdp1csZqu2Dstk3adXJGtF0BWXMXQggxuEDtuQshhBhEwIW7UmqFUmqfUqpYKfUTP7YjSym1Rim1WylVqJS617z/l0qpI0qp7ebHZX5oW4lSaqf58/PN++KVUh8ppYrMf+P80K4pvc7LdqVUs1LqPn+cM6XUX5VSNUqpXb3u83qOlOEx8zm3Qyk1d5Tb9Xul1F7zZ7+plIo1789RSh3rdd6eGOV2+fy7KaV+ap6vfUqpS0aqXYO0bWWvdpUopbab94/mOfOVEaPzPNNaB8wHYAUOALmAAygA8vzUljRgrnk7CtgP5AG/BH7k5/NUAiT2u+//AD8xb/8E+N0Y+FtWAeP8cc6ApcBcYNeJzhFwGfAeoIBFwMZRbtfFgM28/bte7crpfZwfzpfXv5v5/6AACAHGm/9nraPZtn6PPww85Idz5isjRuV5Fmg99wVAsdb6oNa6C3gZuMofDdFaV2qtt5q3W4A9wFjepuUq4Fnz9rPAV/3YFoALgANa61OdyHZatNafAQ397vZ1jq4CntOGDUCsUipttNqltf5Qa+00P90AZI7Ezz7Zdg3iKuBlrXWn1voQUIzxf3fU26aUUsB1wEsj9fN9GSQjRuV5FmjhngGU9fq8nDEQqEqpHGAOsNG86/vm26q/+qP8AWjgQ6XUFqXUneZ9KVrrSvN2FZDih3b1dgN9/8P5+5yB73M0lp53t2H07tzGK6W2KaXWKqXO80N7vP3dxtL5Og+o1loX9bpv1M9Zv4wYledZoIX7mKOUigReB+7TWjcD/wtMAGYDlRhvCUfbuVrrucClwPeUUkt7P6iN94B+GyallHIAVwKvmneNhXPWh7/PkTdKqQcBJ/CCeVclkK21ngP8K/CiUip6FJs05v5uXtxI307EqJ8zLxnhMZLPs0AL9yNAVq/PM837/EIpZcf4o72gtX4DQGtdrbXu0Vq7gD8zgm9HfdFaHzH/rQHeNNtQ7X6LZ/5bM9rt6uVSYKvWuhrGxjkz+TpHfn/eKaW+BVwO3GwGAmbZo968vQWjtj15tNo0yN/N7+cLQCllA64BVrrvG+1z5i0jGKXnWaCF+2ZgklJqvNn7uwFY5Y+GmLW8p4E9WutHet3fu0Z2NbCr/9eOcLsilFJR7tsYF+N2YZynb5qHfRN4azTb1U+f3pS/z1kvvs7RKuBWczTDIqCp19vqEaeUWgHcD1yptW7vdX+SUspq3s4FJgEHR7Fdvv5uq4AblFIhSqnxZrs2jVa7erkQ2Ku1LnffMZrnzFdGMFrPs9G4ajycHxhXlPdjvOI+6Md2nIvxdmoHsN38uAx4Hthp3r8KSBvlduVijFQoAArd5whIAFYDRcDHQLyfzlsEUA/E9Lpv1M8ZxotLJdCNUdu83dc5whi98EfzObcTmD/K7SrGqMW6n2dPmMdea/6NtwNbgStGuV0+/27Ag+b52gdcOtp/S/P+Z4C7+x07mufMV0aMyvNMZqgKIUQQCrSyjBBCiCGQcBdCiCAk4S6EEEFIwl0IIYKQhLsQQgQhCXchhAhCEu5CCBGEJNyFECII/X8G22kkoz3nngAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import matplotlib.ticker as ticker\n",
    "%matplotlib inline\n",
    "\n",
    "plt.figure()\n",
    "plt.plot(all_losses)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 测试"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "There, good but, sir, in that resting may sir.\n",
      "\n",
      "MARDANETA:\n",
      "Wife, in the have his prince othere, as not;\n",
      "My my pleast beir encent but thy lee the day\n",
      "Wine and a isine.\n",
      "\n",
      "FLYOMO:\n",
      "My good the new:\n",
      "Whol of h\n"
     ]
    }
   ],
   "source": [
    "print(evaluate('Th', 200, temperature=0.8))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "There the the shall shall the shall shall the shall be the change the die a shall the shall the should that the shall shall shall be a shall the truther the chander the chander the command.\n",
      "\n",
      "CAMILLO:\n",
      "Wh\n"
     ]
    }
   ],
   "source": [
    "print(evaluate('Th', 200, temperature=0.2))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "py3.6-env",
   "language": "python",
   "name": "py3.6-env"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
